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Abstract 

In this paper , we first briefly overview the update of 
the Self-Mobile Space Manipulator (SM 2 ) configura- 
tion and testbed . The new robot is capable of project- 
ing cameras anywhere interior or exterior of SSF, and 
will be an ideal tool for inspecting connectors, struc- 
tures, and other facilities on SSF. Experiments have 
been performed under two gravity compensation sys- 
tems and a full-scale model of a segment of the Space 
Station Freedom (SSF). This paper then presents a 
real-time shared control architecture that enables the ' s 
robot to coordinate autonomous locomotion and teleop- 
eration input for reliable walking on SSF. Autonomous 
locomotion can be executed based on a CAD model and M 
off-line trajectory planning, or can be guided by a vi- 
sion system with neural network identification. Tele- 
operation control can be specified by a real-time graph- 
ical interface and a free-flying hand controller. SM 2 
will be a valuable assistant for astronauts in inspection 
and other EVA missions. 


1 INTRODUCTION 

Since 1989, we have been developing the Self Mobile 
Space Manipulator (SM 2 ) which is a walking robot 
to assist astronauts on the Space Station Freedom 
and other space structures in performing construction, 
maintenance and inspection tasks. It has end-effectors 
for attachment, and can step from point to point to 
move freely around the exterior of space structures. 
SM 2 can replace EVA astronauts in performing te- 
dious or dangerous tasks, and can be deployed quickly 
to investigate emergency situations. It is simple and 
modular in construction to maximize reliability, sim- 
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plify repairs and minimize development time. SM 2 is 
lightweight, so it can operate with minimum energy 
and disturbance to the structures. 

Over the past four years, SM 2 has progressed from 
concept, through hardware design and construction, 
to software development and experiments with several 
versions of the robot. During the first year, we devel- 
oped a concept for robot mobility on the space station 
trusswork, and experimentally tested a variety of con- 
trol algorithms for simple one-, two- and three-joint 
robots. During the second year, we developed a sim- 
ple, five-joint robot that walked on the tubular-strut- 
and-node structure of the original Space Station Free- 
dom design, and a gravity compensation system that 
allowed realistic testing in a simulated zero-gravity en- 
vironment. The third-year work focused on develop- 
ment of the manipulation function; we added a part- 
gripper and extra joint at each end of the robot, and 
developed related control software. 

In this paper, we will report the research and devel- 
opment work performed during the forth year of the 
project, with emphasis on the shared control system 
developed to facilitate the execution of complex tasks 
in space applications. 


2 NEW SM 2 DEVELOPMENT 

In response to the changing design and needs of 
SSF, our focus has shifted to adapting SM 2 as a mo- 
bile inspection robot to augment the fixed video cam- 
eras planned for SSF. The robot’s size and configu- 
ration have been adjusted to accommodate the new 
truss structure. The space station truss design has 
been changed by NASA in favor of the current pre- 
integrated truss (PIT) design, utilizing I-beam mem- 
bers. The new truss design is hexagonal, rather than 
rectangular in shape. Therefore, our first goal was to 
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modify the SM 2 configuration to adapt to this new 
space station truss. 

The second goal of the project was to specialize the 
SM 2 robot as an inspection robot. There is a vital 
need for inspection of facilities on the space station, 
such as fluid connectors, electric cables, and bolted 
segments. Able to reach both exterior and interior of 
the space station, the movable cameras will be essen- 
tial for this task. SM 2 will be capable of projecting 
cameras to any position on the space station through 
its inherent self-mobility. 

2.1 Robot Configuration 

The robot’s size and configuration have been ad- 
justed for the new truss structure as shown in Fig- 
ure 1. On the previous truss design, five degrees of 
freedom (DOF’s) were sufficient for locomotion from 
any given node to any adjacent node. The robot had 
two joints at each tip and one elbow joint. In order 
to enable the new robot to step from one face of the 
redesigned hexagonal PIT structure to adjacent faces, 
and to retain the symmetry of the SM 2 , the new robot 
requires a total of seven joints, three at each tip and 
one at the elbow. The symmetry of the robot mech- 
anism is important for the control of locomotion, so 
that as the base of the robot is switched, we simply 
switch the numbering of the joints from the base to 
the tip. This allows the out-of-plane motion needed 
to step from one face of the truss to another. In addi- 
tion, the total length of the robot has been increased, 
and the flexibility of the two long links has been re- 
duced so as to accommodate the size of the new truss 
design, while still maintaining the low mass essential 
for space applications. 

Each of the seven joints is identical, self-contained 
and modular so that a minimum inventory of parts is 
required for joint repair or replacement. The joints are 
driven by harmonic motors and are wired in a modular 
fashion so that only one 16-pin connector is required 
to deliver all signals and power to each of the joints. 

2.2 Beam Grippers 

The new truss structure made the old node grip- 
pers obsolete and required design of new grippers that 
could attach to the aluminum I-beams of the PIT 
structure. Each end of the SM 2 is now equipped with 
a three-fingered gripper capable of grasping I-beam 
flanges of various thickness and width, as shown in 
Figure 2. The single finger, driven by a DC motor, 
slides back and forth to allow opening and closing of 


the gripper. A linear potentiometer measures the sin- 
gle finger position, while motor current indicates grasp 
force. 

Each gripper has been equipped with sensors neces- 
sary for reliably and securely grasping the beam. Us- 
ing force-sensing resistors, contact switches on each of 
the three fingers can be checked to verify a good grasp. 
In addition, capacitive proximity sensors at the base 
of the fingers sense beam proximity up to about four 
inches away and are useful in aligning the gripper with 
the beam. 

2.3 Cameras Modules 

There are three camera modules attached to the 
robot, one at each tip, and one on the elbow joint. 
Each camera has separate controllable zoom, fo- 
cus, and iris with four high-intensity lamps arranged 
around each camera. 

The elbow camera has one motorized degree of free- 
dom. Since the robot has one redundant DOF, the el- 
bow camera has effectively two DOFs in determining 
it’s view. With both ends of the robot attached to the 
truss, for example, the collection of all possible views 
sweeps out a half torus about an axis defined by the 
two base joints at each tip. Thus, the elbow camera 
can provide valuable visual information about global 
location on the space station. 

The two tip cameras serve twin purposes. The pri- 
mary purpose is, of course, visual inspection by hu- 
man operators. The robot tip camera at the free 
end can provide views of the truss structure that any 
fixed camera around the space station simply cannot 
achieve. I-beam connections as well as the inside faces 
of the I-beams are two locations where a movable cam- 
era might provide significantly better views. The sec- 
ondary purpose for the cameras concerns autonomous 
locomotion on the truss. We use neural-network based 
machine vision with images from the tip camera to 
autonomously mate the gripper to the I-beam flanges. 
The tip camera module and end-effector are shown in 
Figure 3. 

2.4 Gravity Compensation 

To simulate the zero gravity environment of space, 
we use two independent gravity compensation systems 
developed at Carnegie Mellon University. Each grav- 
ity compensation system provides a constant upward 
vertical force through a counterweight mechanism and 
a series of cable and pulleys. The support cables are 
attached to the centers of gravity of the two long links 
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on the robot. A 10:1 ratio in the counterweight mech- 
anism keeps the increased inertia in the vertical direc- 
tion to 10 

The support cables attached to the robot are 
tracked overhead by two separate, actively controlled 
carriage systems. Angle sensors detect x-y movement 
of the support cables. The first system is a Cartesian 
gantry system and allows robot motion in an area that 
is 17 feet long and 9 feet wide. This allows us to test 
large global stepping motions for the robot. The sec- 
ond system is a smaller cylindrical compensation sys- 
tem supporting a smaller field of motion. This allows 
a large variety of motions to be tested without the 
supporting cable of the larger system interfering with 
the carriage beam of the smaller system [2]. 

In addition to the mechanical gravity compensa- 
tion, we provide for active residual gravity compensa- 
tion in software to correct for minor discrepancies in 
the mechanical system. This is especially necessary to 
provide appropriate torques for the three joints at the 
free end of the robot. The combination of mechanical 
and active gravity compensation provides for realistic 
zero gravity experiments and testing. 

2,5 Truss Mock-up 

In our lab, we have built a truss mock-up which is 
a full-scale representation of a small portion of the en- 
tire truss structure on the space station. The mock-up 
includes four faces of the hexagonal structure as shown 
in Figure 4. Each beam is constructed of wood with 
sheet aluminum laminated to the flange faces to al- 
low for realistic machine vision testing. Varying flange 
widths and thicknesses allow for robust testing of the 
grippers. 


3 REAL-TIME SHARED CONTROL 
ARCHITECTURE 

At the heart of the SM 2 control software lies a real- 
time shared control architecture [1]. It is modular in 
design whereby tasks are composed of independent, 
reusable subtasks. High level tasks for the SM 2 robot 
range from teleoperation to semi- autonomous tasks to 
fully autonomous walking. These tasks often use many 
of the same subtasks such as trajectory tracking, beam 
grasping, point convergence, and switching the base 
of the robot. These subtasks are coded as modular 
library routines which may be dynamically sequenced 
through a coordination module and state machine. 


3.1 Coordination of Tasks 

The various task modules need to be coordinated 
in an intelligent fashion. We used a state machine, 
programmable through a simple language and parsed 
in real-time. The state file describes the following at- 
tributes of the state machine: 

• Defines the number of subtasks and the possible 
message inputs and outputs for each subtask. 

• Defines all tasks (states). 

• Defines all possible transitions and the initial task 
(state). 

A subtask is defined as shown in the following ex- 
ample: 

SUBTASK grasp 

INPUT on off open close stop gripper 1 
gripperS 

OUTPUT noncontact contact done grabbed 
The first line merely assigns a label to the subtask. 
The second line gives a list of valid messages that 
the subtask grasp will accept as input. Each of these 
inputs is easily understood. For example open com- 
mands the subtask to open the gripper, while gripperS 
commands the subtask to switch to gripper2. Finally, 
the last line specifies the outputs of the subtask. These 
are then used in the sequencing of states. 

A typical task specification might appear as follows: 

TASK tele.gripper.close 

SUBTASKS grasp tele 
START tele:on tele:grp grasp:close 

END grasp:off 

Here, again, the first line merely assigns a label to 
the task. The second line specifies which subtasks are 
part of the overall task. In this example, both grasp 
and teleoperation combine to form the specific task. 
The next line specifies what messages to send to the 
various subtasks at the start of the overall task. The 
first two commands make certain that teleoperation 
is in the on mode and that the control mode is the 
gripper mode. The final start message instructs the 
grasp subtask to attempt to close the gripper. In the 
final line, we specify what messages to send at the end 
of a task execution. Once the gripper is closed, we 
instruct the subtask grasp to turn off. 

Finally, below we show an example of specifying 
state transitions and an initial state: 

TRANSITION tele.down tele.gripper.idle 
tele. gripper. close 
INITIAL.TASK tele.init 
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The transition statement simply states that when 
the subtask tele receives a down message - when the 
appropriate button is pressed on the teleoperation 
hand controller - the state machine should sequence 
from the idle gripper mode to the close gripper mode. 

In such a manner, high-level tasks can quickly be 
programmed from a library of subtasks through the 
state machine. Note that subtasks are reusable from 
state to state and can be switched on and off when 
necessary. For example, the grasp subtask is equally 
necessary in the autonomous locomotion mode as well 
as the teleoperation mode. 

In short, the state machine allows subtasks to 
be shared by high-level tasks which can be rapidly 
re-programmed with minimal re-coding and no re- 
compilation. This allows for elegant and rapid soft- 
ware development. 

3.2 Task Modules 

We have developed several reusable task modules 
for the SM 2 control software. In each control cycle, 
the task modules perform four basic functions: 

• Read messages from the state machine and re- 
spond in appropriate fashion. 

• Read sensor devices, global variables, or receive 
input from remote tasks. 

• Generate desirable control motion based on local 
inputs. 

• Send appropriate messages to the state machine. 

Since each subtask module produces desired con- 
trol commands based solely on its limited criteria, one 
module - the combination module - is required to in- 
telligently combine these desired control outputs from 
individual task modules into one coherent control sig- 
nal. The combination module therefore ensures rea- 
sonable control outputs based on a weighted average of 
the control commands of the individual task modules. 

Remote task modules do not fundamentally dif- 
fer from other modules except in one respect. These 
modules are run on a separate workstation or pro- 
cessing board, usually due to high computational re- 
quirements that cannot be met in real time. These 
modules can interface with the slower real-time boards 
via UNIX sockets, a VME bus, or serial lines. Menu- 
driven user interfaces as well as a real-time graphical 
displays are two examples of such computationally in- 
tensive remote tasks. These, along with the other task 


modules will be discussed in the context of the follow- 
ing two sections which discuss (1) autonomous walking 
on the truss, (2) and teleoperation. 

4 AUTONOMOUS LOCOMOTION 

4.1 Model-Based Walking 

The operating environment for the SM 2 is very 
structured and can easily be modelled with a great de- 
gree of accuracy. Hence, it is possible for the robot to 
execute a pre-planned sequence of walking steps based 
solely on a model of the space station truss struc- 
ture. We have successfully executed various sequences 
of four steps on the truss mock-up, including steps 
of variable length and between different faces of the 
hexagonal space station truss structure. Each walking 
step is decomposed into several distinct phases: (1) 
ungrasping the beam, (2) separating smoothly from 
the beam, (3) executing a global trajectory, (4) exe- 
cuting a straight-line motion towards the beam, (5) 
closing the gripper, and (6) switching the base for the 
next step. 

First, the gripper is opened until the sliding po- 
tentiometer indicates that the gripper is in the fully 
opened position. Second, while keeping the orienta- 
tion of the gripper aligned with the beam, the free 
end is moved above the beam in a straight-line mo- 
tion so as to avoid potential collisions with the space 
station truss. Once the free end is safely above the 
truss structure, control is switched to the execution of 
a global trajectory in the state machine. 

A global trajectory is defined minimally by the 
starting point and the target destination. The opera- 
tor, however, is free to include as many via points as he 
chooses along the path of the trajectory. These points 
may be generated alternatively in a preprogrammed 
file or through the real-time graphical display as dis- 
cussed in the subsequent section. As the trajectory 
is being executed, errors are dynamically corrected by 
continuously calculating a smooth path between the 
current position and the desired trajectory path. If 
no, intermediate points are specified along the tra- 
jectory, the inverse kinematic algorithm, as explained 
later on, will generate intermediate points which lead 
to a smooth trajectory. 

The trajectory will finish with the proper gripper 
orientation about 20 inches above the target beam and 
location. From there, the state machine enters the 
next phase of execution; that is, a straight line descent 
towards the target beam along the surface normal of 
the beam. 
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Each gripper has multiple sensors that can be used 
during approach to the beam and grasping. Proximity 
sensors at the base of each finger provide information 
about the relative orientation of the gripper and beam 
from several inches away, and signal when the gripper 
face is close against the beam. Contact switches, us- 
ing force sensing resistors (Interlink), sense contact of 
the three fingers with the edge of the beam to verify 
a sense grasp. Gripper motor current is also sensed 
to indicate the grasp force. After the initial grasp is 
made, the gripper is opened slightly (about 0.25 inch) 
and closed again. This helps to automatically correct 
for any remaining misalignment. 

Finally, if another step is to follow, the robot will 
switch bases. What was the free end before, will now 
become the fixed base and vice versa. 

It is important to note that the entire sequence de- 
scribed above is controlled through the state machine. 
Each phase of the stepping motion will execute only 
when the appropriate done message is sent by the con- 
trol software to the state machine. The proper done 
message triggers a transition to the next state. The 
entire walking step is divided into a sufficient number 
of subtasks, any or all of which can be used during 
other modes, such as teleoperated or semi-autonomous 
control. 

4.2 Neural Network Based Visual Servo- 
ing 

Although we have a good model of the environment, 
errors can accumulate over consecutive steps. This 
can potentially lead to a failure in properly grasping 
the next beam. If this should occur, a neural-network 
based vision system will assume control, correct any 
such error and properly complete the grasping of the 
beam. It is preferable to use the vision system only 
when failing to complete a grasp, since the vision sys- 
tem slows the system performance significantly. The 
main bottleneck is, of course, the acquisition of the 
images at a high rate. 

We trained a neural network on 40x40 digitized im- 
ages of flanges at various translational offsets, heights, 
and rotations. The neural network learned through 
the standard back propagation learning algorithm. 

Once the vision system has placed the gripper in 
contact with the beam, the state machine returns con- 
trol to the same states and subtasks used for closing 
the gripper as mentioned previously. 

Unlike the previous strut-and-node design of the 
space station truss structure, the current design causes 
uncertainty in the location of the robot on the truss 


structure, since SM 2 is free to grasp the beam any- 
where along its length. That uncertainty could po- 
tentially be periodically removed by using the vision 
system to locate certain known special locations on the 
space station truss. One such special feature might be 
where two or more beams join. Further work needs to 
be done in this direction. 


5 TELEOPERATION 

We have developed two different methods for tele- 
operation. The first method utilizes a six-DOF hand 
controller to guide the free end of the robot. The 
second method utilizes the real-time graphics display 
which provides two views of the space station truss 
structure. By selecting the target location for the 
robot arm with a mouse, the robot can be made to 
execute large global trajectories. 

5.1 Hand Controller 

We use a commercial, six-DOF, free-flying hand 
controller as the principal means for teleoperated con- 
trol. The device, called the Bird y operates with a sta- 
tionary radio transmitter and a moving receiver. Both 
the position and orientation of the receiver relative to 
the transmitter is communicated via a serial line to 
the controller at a rate of 10Hz. The moving receiver 
is attached to a cylindrical stick with an enable switch 
controlled with the thumb, and another multi-purpose 
two-way switch controlled with the index finger. Fig- 
ure 5 shows the control station configuration and the 
use of the hand controller. 

The hand-controller is used in conjunction with a 
graphical user interface to determined the mode of op- 
eration for the hand controller as well as the function 
of the two-way switch. The menu-driven user interface 
allows the operator to select one of three basic modes 
of operation, as well as which end of the robot is the 
active one. The three modes are (1) position control, 
(2) velocity control, and (3) gripper control. 

In gripper mode, the two-way switch controls the 
opening and closing of the gripper. Velocity control 
is generally used during large global motions of the 
robot, while position and gripper control are used 
when grasping a beam and switching the fixed base 
of the robot. 

In each mode, the operator can select whether 
the motion of the free end of the robot is to be 
base-relative, tip-relative, or semi- autonomous. Tip- 
relative motion is generally the most useful when the 
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only visual feedback for the operator is from the el- 
bow and tip camera (i.e. the robot itself is hidden 
from view). Base-relative motion is useful in conjunc- 
tion with either fixed camera views or the real-time 
graphical display which reveal the global position of 
SM 2 on the space station truss. 

In manually mating the free end of the robot to 
one of the I-beam flanges, the semi-autonomous mode 
simplifies the process for the operator. The semi- 
autonomous mode allows the operator to automati- 
cally orient the free gripper to the correct orientation 
for grasping the beam. The control software utilizes 
knowledge of which beam the fixed end is currently 
attached to and which beam the operator wishes to 
grasp in order to select the proper orientation for the 
gripper. With this semi-autonomous orienting, the 
process of teleoperated walking on the space station 
truss is significantly facilitated. Requiring only mini- 
mal training, we have repeatedly demonstrated teleop- 
erated walking on the truss mock-up, with and with- 
out the robot in view of the operator. 

The above discussion illustrates several dimensions 
of the shared control architecture. We achieve a blend 
of teleoperation and autonomous locomotion with- 
out the need for new software code. In the semi- 
autonomous teleoperated mode, we use the same sub- 
task to achieve the proper orientation of the gripper 
before grasping as we do in autonomous walking. Fur- 
thermore, we are able to use the same grasping subtask 
for autonomous walking and teleoperation. In fact, 
the message to the state machine issued during au- 
tonomous walking and teleoperated control is exactly 
the same: close gripper. Thus, all the safety precau- 
tions used for ensuring a secure grasp of the beam 
during autonomous walking are automatically incor- 
porated when the operator commands the gripper to 
close on the beam. 

In another example, the operator may wish to in- 
spect the length of a beam. Rather than worry about 
following a precise straight line with the hand con- 
troller, the operator may wish to surrender control of 
one directional degree of freedom (transverse to the 
beam) so that he can inspect the length of the beam 
with variable speed, approaching the beam closer if 
some damage is observed. This may be achieved by 
employing the same trajectory subtask as is used for 
the autonomous walking. Again, the shared control 
architecture allows an elegant merging of autonomous 
and teleoperated function. Simply with some minor 
additions to the state machine, the teleoperation func- 
tion is seamlessly incorporated into the overall control 
architecture. 


5.2 Real-Time Graphical Interface 

Rather than explicitly define the trajectory which 
the robot is to follow, an operator may wish to simply 
specify starting and stopping points for global step- 
ping motions. To this end, we have developed a real- 
time graphical interface. 

The graphical user interface is a PHIGS and 
XView-based application which runs as a remote task 
module. It has been designed to perform the following 
functions: 

• It provides a 3D display of the robot position, con- 
figuration, and its location on the space station 
truss structure. Ambiguities in the 3D display 
on the 2D screen are resolved by providing two 
separate, modifiable views. 

• It allows for manually controlling task sequencing 
in the state machine in real-time. 

• It serves as a teleoperation input device for con- 
trolling global robot motions. 

• It allows for visually pre-planning and simulating 
robot stepping motions to avoid obstacles and sin- 
gular or near singular configurations. 

• It serves as visual feedback to an operator by pro- 
viding a global view of the robot on the space 
station truss. In addition, it warns of potential 
collisions by sending appropriate messages to the 
state machine. The operator can thus modify the 
robot trajectory accordingly. 

In teleoperation mode, the graphical display trans- 
lates mouse commands into trajectories in real-time. 
Once again, teleoperation and autonomous function 
are combined through the shared control structure. 
After the operator specifies desired steps for the robot, 
the same subtasks which perform autonomous walking 
are employed. 

6 CONCLUSION 

The SM 2 robot has been redesigned to be compat- 
ible with the new space station truss structure. Both 
the software and hardware of the SM 2 system has 
been designed to be modular, in order to shorten re- 
pair, maintenance, and development time. We have 
demonstrated both autonomous walking as well as 
teleoperation functions in a single shared control ar- 
chitecture. Depending on the calibration errors, the 
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model-based locomotion with off-line trajectory plan- 
ning, and neural- network based vision can be used for 
reliable walking. The real-time graphics interface pro- 
vides a valuable tool for specifying control inputs in 
teleoperation and for displaying the robot configura- 
tion under communication delay. The free-flying hand 
controller provides an easy way to command robot ac- 
tion with two monitor views from the robot cameras. 
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Figure 3: The tip camera module and beam gripper 
serve as vision guided effector in stepping motion 


Figure 1: A 7-DOF robot manipulator for space sta- 
tion inspection 
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Figure 4: The Space Station Freedom mock-up and 
telerobotics testbed 



Figure 5: The control station with a free-floating hand 
controller 
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